Please Do Not Return Your Form to the above Address. 1. Report Date (dd-mm-yyyy)

This report summarizes the important research activities, study results and research accomplishments out of the RMCLM project in the past two-year period. The objective of this project includes research of the fundamental issues related to the use of multiple input modalities and their fusion to enable robust and automatic cognitive load measurement (CLM) in the real world. Firstly, we carried out a further literature review on physiological measures of cognitive workload to include the recent advances of physiological measures of cognitive workload. In the meantime, we examined the use of various features (e.g. spectral and approximate entropies, wavelet-based complexity measures, correlation dimension, Hurst exponent) of electroencephalogram (EEG) signals to evaluate changes in working memory load during the performance of a cognitive task with varying difficulty/load levels. Eye based CLM was also studied. Eye activities such as pupillary response, blink, and eye movement (fixation and saccade) were investigated for CLM. We further investigated the linguistic and grammatical feature based CLM in this study and analyzed novel linguistic features as potential indices of cognitive load. Other modalities such as Galvanic skin response (GSR), face, and writing behavior were also extensively analyzed in indexing cognitive load levels. We also investigated the effect of stress on cognitive load. All together, we had carried out CLM study of multiple unobtrusive modalities, namely EEG, eye activity, linguistic and grammatical features, GSR, face, and writing behavior for CLM as well as emotion interference for CLM, in the past two-year period. List of Publications a) Papers published in peer-reviewed journals 1. Khawaja, M. A., Chen, F., and Marcus, N., “Measuring Cognitive Load using Linguistic Features Implications for Usability Evaluation and Adaptive Interaction Design”, International Journal of Human-Computer Interaction, vol. 30, no. 5, pp. 343-368, 2014. 2. Hussain, M. S., Calvo, R. A., and Chen, F., “Automatic cognitive load detecting from face, physiology, task performance and fusion during affective interference”, Interacting with Computers, vol. 25, no. 4, 2013. Elsevier. 3. Chen, S., and Epps, J., “Blink Analysis for Cognitive Load Estimation: Towards Wearable Computing that Understands Your Current Task”, IEEE Pervasive Computing, vol.12, no.3, 56-65, 2013. 4. Zarjam, P., Epps, J., Chen, F., and Lovell, N. H., “Estimating cognitive workload using wavelet entropy-based features during an arithmetic task”, Computers in Biology and Medicine, vol. 43, no.12, pp. 2186-2195, 2013. Elsevier. 5. Chen, F., Ruiz, N., Choi, E., Epps, J., Khawaja, A., Taib, R., Yin, B. and Wang, Y., “Multimodal Behaviour and Interaction as Indicators of Cognitive Load”, ACM Transactions on Interactive Intelligent Systems, vol. 2, no. 4, article 22, December 2012. Distribution A: Approved for public release; distribution is unlimited. FA2386-12-1-4049 Final Report Robust Multimodal Cognitive Load Measurement 2 6. Khawaja, M. A., Chen, F., Marcus, N., “Analysis of Collaborative Communication for Linguistic Cues of Cognitive Load”, International Journal of Human Factors and Ergonomic Society, vol. 54, no 4. pp 518-529, August 2012. 7. Chen, S., Epps, J., “Automatic Classification of Eye Activity for Cognitive Load Measurement with Emotion Interference”, Computer Methods and Programs in Biomedicine, 2012. b) Papers published in peer-reviewed conference proceedings: 8. Chen, S., Epps, J., Chen, F., “An Investigation of Pupil-based Cognitive Load Measurement with Low Cost Infrared Webcam under Light Reflex Interference”, Proc. EMBC 2013, Osaka, Japan, 2013. 9. Wang, W., Li, Z., Wang, Y., and Chen, F., “Indexing cognitive workload based on pupillary response under luminance and emotional changes”, Proc. Int‟l Conf. Intelligent User Interfaces (IUI), pp. 247-256, 2013. 10. Nourbakhsh, N., Wang, Y., and Chen, F., “GSR and blink features for cognitive load classification”, Proc. IFIP Conference on Human-Computer Interaction (INTERACT), 2013. 11. Conway, D., Dick, I., Li, Z., Wang, Y., and Chen, F., “The effect of stress on cognitive load measurement”, Proc. IFIP Conference on Human-Computer Interaction (INTERACT), 2013. 12. Yu, K., Epps, J., and Chen, F., “Mental Workload Classification via Online Writing Features”, Proc. Int‟l Conf. Document Analysis & Recognition (ICDAR) 2013. 13. Zarjam, P., Epps, J., Chen, F., and Lovell, N. H., “Classification of Working Memory Load Using Wavelet Complexity Features of EEG Signals”, Lecture Notes in Computer Science, vol. 7664, pp. 692–699, Nov. 2012, Springer-Verlag Berlin. 14. Zarjam, P., Epps, J., Lovell, N. H., and Chen, F., “Characterization of Memory Load in an Arithmetic Task using Non-Linear Analysis of EEG Signals”, Proc. of the 34th IEEE Engineering in Medicine and Biology Conference (EMBC'2012), pp. 3519-3522, California, USA, 2012. Distribution A: Approved for public release; distribution is unlimited. FA2386-12-1-4049 Final Report Robust Multimodal Cognitive Load Measurement